Control of a unit provided with a processor

ABSTRACT

The invention relates to a method for controlling a unit provided with a processor. The method comprises receiving at least one graphical notation in the form of positions representing a sensor device&#39;s movement across a base that is provided with a position-coding pattern, while the graphical notation was made, identifying, based on the at least one graphical notation, at least one command for the unit provided with a processor, receiving an address to the unit provided with a processor, and controlling the unit provided with a processor by sending the at least one command to the address. The invention further comprises a computer program product and a device for implementing the method, a method for controlling a unit provided with a processor using a sensor device and a product kit for controlling a unit provided with a processor.

This application is a Divisional of co-pending application Ser. No.10/178,734, filed on Jun. 25, 2002, for which priority is claimed under35 U.S.C. §120, which claims priority under 35 U.S.C. §119(a) to SwedishApplication No. 0102236-7 filed in Sweden on Jun. 25, 2001, andProvisional Application No. 60/301,446 under 35 U.S.C. §119(e) filedJun. 29, 2001 the entire contents of all of which are incorporated byreference.

TECHNICAL FIELD

The present invention relates to methods for controlling a unit providedwith a processor, and to a device, a computer program product and aproduct kit for the same purpose.

BACKGROUND ART

Concurrently with the development of information technology, newopportunities for realization of so-called “intelligent homes” arearising. “Intelligent homes” are dwellings, where one or more electronicdevices can be controlled or monitored from units located outside thehouse. Examples of devices that can be controlled or monitored remotelyare heating devices, for example in holiday cottages, fire alarms, floodalarms and burglar alarms. Other examples of devices that could becontrolled remotely or monitored are computers, lighting, irrigationsystems, TV, video, music centers, refrigerators, freezers, cookers,microwave ovens or washing machines. A basic requirement is that thedevice can be provided with or can be connected to a processor which canreceive, process and pass on information. The processor should then becapable of being connected to some form of communication network, suchas the telephone network or the Internet.

For the intelligent home to be controllable from a remote location,there is a need for a remote control unit. The remote control unitaccording to existing systems is typically a telephone, mobile telephoneor computer terminal, which communicates with the processor via thecommunication network. In its simplest form, the remote control unit isa telephone or mobile telephone which communicates directly with theprocessor by means of predefined key commands or codes. In a moreadvanced form, the remote control unit is a computer or mobile telephonewhich communicates with the processor via a computer network, such asthe Internet. A problem with using a telephone as remote controlterminal is that the user interface is based on pressing keys on anumeric keypad, which can lead to problems, for example in learning orremembering the codes that are to be used. Communicating by pressingkeys on a numeric keypad is thus not particularly user-friendly. Inaddition, a mobile telephone has usually limited data entry facilitieson account of its small format. In addition, no or limited feedback isgiven in response to what has been keyed into the mobile telephone.

The use of a computer with Internet connection is a solution which worksproviding such a computer is available. In addition, this requires aspecific user interface to be developed and adapted for the respectiveunit which is to be controlled remotely. In addition, people who are notused to computers perceive solutions which require the use of a computeras complicated and difficult to use.

Another problem is that as the system is developed with more devicesthat can be controlled remotely, demands are made concerning a uniformstandard for their control. For example, it would be time-consuming andthus unsatisfactory to need to look up different Internet home pages inorder to control the central heating and the microwave oven.

There is thus a need for a method for user-friendly and flexible remotecontrol of units provided with a processor, which method utilizes aportable device.

SUMMARY OF THE INVENTION

The object of the present invention is to solve the above problemcompletely or partially.

Methods for fulfilling the object are described in independent claims 1and 30. A computer program product, a device and a product kit forfulfilling the object are described in independent claims 26, 27 and 33,respectively. Embodiments of the invention are apparent from theappended dependent claims and from the following description.

According to a first aspect of the invention, there is provided a methodfor controlling a unit provided with a processor. The method comprisesreceiving at least one graphical notation in the form of positionsrepresenting a sensor device's movement across a base that is providedwith a position-coding pattern, while the graphical notation was made.The method further comprises identifying, based on the at least onegraphical notation, at least one command for the unit provided with aprocessor, and receiving an address to the unit provided with aprocessor. Finally, the method comprises controlling the unit providedwith a processor by sending the at least one command to the address.

A graphical notation may be any writing and/or drawing, which is made ona base, using a sensor device, that records positions based on theposition-coding pattern provided on the base. The graphical notation maybe a single, continuous stroke, or a group of such strokes. Each strokemay be represented as a sequence of coordinate pairs coded by theposition-coding pattern on the base. Thus, the user input may bedigitized without any additional operation on the part of the user, suchas scanning the base or digitizing it in some other way.

According to the present invention, such graphical notations may be usedfor drawing up a command structure or hierarchy on a base, such as apaper. The command structure may comprise commands for controlling aunit provided with a processor. This provides a rapid, simple and easyto understand way for the user to control the unit provided with aprocessor. In addition, by noting down the commands, the user obtainsautomatically an easy to understand copy of what was entered into theunit.

An additional advantage of the present invention is that the user is notlimited to one base which is specific to a particular type of command.

A unit provided with a processor can be a device which contains any formof processor, for example a microprocessor. Examples of such devices arecomputers, modern household appliances (dishwashers, microwave ovens,cookers/stoves, audio/video players, etc), industrial machinery andother computer-controlled applications such as central heatinginstallations, air conditioning installations, telephone systems andmonitoring/alarm systems.

A base can be a device on which information can be noted down, usually asheet of paper, a drawing board or similar medium which is provided witha position-coding pattern which makes possible electronic recording ofwhat is noted down on the base.

A command can be such words, symbols, sub-addresses in computernetworks, program names, command names, file names, storage addresses orsymbols which represent particular operations, functions, operators,parameters and arguments that can be used individually or in combinationfor controlling a unit provided with a processor.

An address of the unit provided with a processor can be an address forcommunication with the unit provided with a processor. It can be acomputer network address, such as a standard IP address, but other formsof address are possible. It can also be an address of a unit via whichthe unit provided with a processor communicates, for example aproxy-server or a unit for short-range communication such as Bluetooth®.Thus it is also possible to utilize the invention when the user is inthe vicinity of the unit provided with a processor without having toconnect, for example, via a computer network. An address can also beexpressed as a short name, which is associated with the address. Forexample, the word “home” can be an indication of a particular computernetwork address of a computer which is located in the user's home.

Making graphical notations may, but does not necessarily, mean that amark is left on the base. This may provide the advantage that the basewill constitute a copy of what was entered into the unit provided with aprocessor. With the reuse of previously entered commands it can,however, be an advantage if no mark is left on the base, in order tomake possible repeated use.

According to one embodiment of the invention, the at least one graphicalnotation may at least partly comprise handwritten characters. Thus, theat least one graphical notation may be at least partly converted into acharacter coded format for identifying the at least one command. Thisprovides a very flexible way of entering commands, since there is noneed for predefining or associating commands with user-friendly symbols.

Alternatively, identifying the at least one command may compriseidentifying at least one graphical symbol from the at least onegraphical notation, the graphical symbol representing the at least onecommand. This way, user friendly symbols, which speed up the making ofthe graphical notation, may be provided. This variant may be combinedwith the above described character coded variant, by e.g. providingpredefined symbols for frequently used commands or addresses, while lessfrequently used commands or addresses are to be noted as handwrittencharacters for conversion to character coded form.

According to another embodiment of the invention, identifying the atleast one command may comprise detecting a command indicator, based onthe at least one graphical notation optionally, identifying the at leastone command may comprise identifying a subarea of the position-codingpattern, the subarea being essentially encircled by the commandindicator. The subarea may be associated with the command.

A command indicator may be a graphically noted indication, which isrecognized by the sensor device as an instruction to identify a command.The instruction may be a symbol and in one embodiment of the invention,the symbol may encircle the command, and thus constitute a frame or anyother drawn shape, which wholly or partially encircles the command. Theframe may have a specific appearance, which is recognized by the sensordevice and thus interpreted as the indication that a command is beingentered.

By indicating that the command is being entered, the sensor device maymore easily be able to interpret a graphical notation as a command.

A subarea is an area of the position-coding pattern on the base, whicharea is delimited by e.g. the command indicator or by some otherrelation to the command, such as a certain area encircling the graphicalnotation. For example, a halo-like area surrounding the command may bedefined as soon as a command is recognized. The subarea may, via theposition-coding pattern, be associated with the command, such that acommand may be identified based on a recording of any pair ofcoordinates that falls within the subarea associated with that command.

By allowing the command indicator to define the subarea that isassociated with the command, and indicate that a command has beenentered, the command indicator may have a dual function: defining thesubarea and indicating that a command is being entered.

Thus, according to one embodiment of the invention, sending the at leastone command to the address may be effected in response to a recording ofa pair of coordinates within the subarea. This enables “reuse” of apreviously noted and recorded command.

According to the invention, the address may be received in differentmanners. One alternative is that receiving the address comprisesreceiving the address from a memory in the sensor device. Thus, theaddress to the unit provided with a processor may be preprogrammed andstored in a memory in the sensor device. According to anotheralternative, receiving the address may comprise identifying the addressbased on the at least one graphical notation. Thus, the address may benoted graphically on the base, recorded by the sensor device andoptionally associated with one or more commands.

According to an embodiment of the invention, an association between theaddress and the at least one command may be identified based on the atleast one graphical notation. The association may indicate therelationship between the commands or the command and the address.

According to one embodiment of the invention, receiving at least onegraphical notation comprises receiving at least three separablegraphical notations representing the address, the at least one commandand the at least one association between the address and the at leastone command. Thus, the graphical notations representing the address, thecommand and the association may be clearly distinguishable from eachother.

The at least one association may be identified based on a separablegraphical notation connecting the graphical notations representing theat least one command and the address, respectively. For example, the atleast one association may be identified as a graphical notation havingessentially the shape of a line extending between the graphicalnotations representing the at least one command and the address.Naturally, associations may also connect two commands, or a command andan address. The associations provide a simple and intuitive way ofrelating commands, addresses, etc. to each other.

According to one embodiment of the invention, there is provided, in amemory in the sensor device, an electronic representation of a commandhierarchy which comprises at least two commands and at least oneassociation, each of the at least two commands and the at least oneassociation being graphically represented on the base by nodes and arcs,respectively, and each of the at least two commands being associatedwith a respective subarea of the position-coding pattern.

A command hierarchy, or a command structure, may be any hierarchy ofcommands for one or more units provided with a processor. A commandhierarchy typically has a root, which may be a command or an address toa unit provided with a processor. Further, the command hierarchy mayhave, but does not necessarily need to have, a plurality of branches,which may also be referred to as nodes, each branch or node, in turn,having a number of sub branches, such that a tree-like structure isformed. The command hierarchy may be large, thus comprising a largenumber of command levels and/or a large number of commands on eachlevel. The nodes may be connected by lines, such that each node has onesuperior node, but may have more than one subordinate node. The linesconnecting the nodes may be interpreted as the associations referred toabove, while the nodes may be addresses or commands for controlling aunit provided with a processor. Naturally, one address may in turn havea number of subaddresses to, e.g. different units provided withprocessors.

The inventive method may include forming at least one command stringbased on the command hierarchy, and controlling the unit provided with aprocessor by sending the command string to the address. A command stringis an instruction for a unit provided with a processor, whichinstruction is built up by more than one command and optionally by anaddress. A command string may include more than one command.

Furthermore, identifying the at least one command may comprise receivinga pair of coordinates from one of the subareas, the pair of coordinatesrepresenting a chosen command and identifying an electronicrepresentation of the chosen command, wherein the command string isformed based on the chosen command and at least one hierarchicallysuperior command.

The command hierarchy according to this embodiment may be represented ona base provided with a position-coding pattern. The electronicrepresentation of the command hierarchy may be at least partly providedthrough identifying the command hierarchy based on the at least onegraphical notation. At least partly means that an existing commandhierarchy, which is either preprinted or e.g. graphically noted by theuser or someone else, may be expanded by the user adding furthercommands or addresses.

Alternatively, providing the electronic representation of the commandhierarchy may at least partly comprise electronically receiving theelectronic representation of the command hierarchy. The base, on whichthe graphical notation is made, may be provided with a graphicalrepresentation of at least a part of the command hierarchy. Thus, a basehaving a preprinted command structure may be provided together with anelectronic version of the command structure, which is to be stored inthe memory of the sensor device for future use.

According to the invention, providing the electronic representation ofthe command hierarchy may also comprise identifying, based on the atleast one graphical notation, at least one further command and at leastone further association, and storing, in a memory of the sensor device,the at least one command and the least one further command, based on theat least one further association. Thus, the predefined command structuremay be expanded by the user adding e.g. further commands or by the useradding e.g. an address. The command hierarchy may also be used byrecording pairs of coordinates within predefined subareas, which areassociated with commands in the command hierarchy. Based on suchrecordings, commands may be sent to the unit provided with a processor,as described above.

The electronic representation of the command hierarchy may be stored ina tree data structure in the memory of the sensor device. A tree datastructure may be any data structure for representing a hierarchy or atree structure. Numerous such data structures are known. According tothis alternative, command strings are formed in response to anindication of a command, such as a recording of a pair of coordinateswithin a subarea that is associated with a command. The command stringmay be built up starting with the selected command and adding eachhierarchically superior command, until the root is reached. Building thecommand string may also comprise adding separating characters, such as“\” etc., arranging the commands in a suitable order and adding thenecessary parameters or switches.

Alternatively, the electronic representation of the command hierarchymay be stored in the form of at least one command string which isformable based on the electronic representation of the commandhierarchy. Thus, a plurality, or all, of the command strings that may beformed based on the command hierarchy may be stored in the form of moreor less complete command strings. When a command is indicated by e.g. arecording of a pair of coordinates within a subarea, the command stringcomprising that command is retrieved and sent to the unit provided witha processor.

According to a second aspect of the invention, there is provided acomputer program product for controlling a unit provided with aprocessor. The computer program product comprises instructions for asensor device, which, when executed, causes the sensor device to performthe above described method.

According to a third aspect of the invention, there is provided a sensordevice for controlling a unit provided with a processor. The sensordevice comprises a signal processor for receiving positions representingthe sensor device's movement across a base that is provided with aposition-coding pattern. The signal processor is arranged for receivingat least one graphical notation in the form of the positions,identifying, based on the at least one graphical notation, at least onecommand for the unit provided with a processor, receiving an address tothe unit provided with a processor, and controlling the unit providedwith a processor by sending the at least one command to the address. Thesignal processor of the sensor device may be arranged to perform themethod described above. The method may be implemented by means ofspecial-purpose circuitry, by programmable microprocessors or by acombination thereof.

According to a fourth aspect of the invention, there is provided amethod for controlling a unit provided with a processor. The methodcomprises using a sensor device for recording positions representing thesensor device's movement across a base provided with a position-codingpattern, while a graphical notation is made on the base, notinggraphically on the base, using the sensor device, at least one commandto the unit provided with a processor, and sending the command to theunit provided with a processor for controlling this unit. This methodprovides a user-friendly and intuitive way of controlling a unitprovided with a processor.

The method can, for example, be used when a user wants to enter acommand string into, for example, a computer or other unit provided witha processor connected to the sensor device, e.g. by wireless means. Themethod enables a unit provided with a processor to be controlled withoutusing a preexisting user interface, such as a preprinted base providedwith command options. Instead, the user may use any base that isprovided with a position-coding pattern, on which he or she makesgraphical notations, which correspond to the desirable commands andsends the commands to the unit provided with a processor.

According to a fifth aspect of the invention, there is provided aproduct kit for controlling a unit provided with a processor. Theproduct kit comprises a control base, provided with a position-codingpattern, on which control base a command hierarchy, comprising at leasttwo commands, is graphically represented, and on which control base eachof the at least two commands is associated with a respective subarea ofthe position-coding pattern, and a computer program product comprisingan electronic representation of the command hierarchy, whereby the atleast two commands are identifiable based on a recording of a pair ofcoordinates within its respective associated subarea. Such a product kitprovides a way of integrating e.g. a household appliance into a remotecontrol system for an intelligent home.

The electronic representation of the command hierarchy may be stored inthe memory of the sensor device, and possibly completed by the useradding an address to which the commands are to be sent, therebycompleting the installation of the appliance in the intelligent home.The user may then either use the control base provided for selectingpredefined commands, or draw up the command hierarchy on an arbitrarybase and send a command string to the unit provided with a processor.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be described in greater detail in the following withreference to the attached schematic drawings which, for the purposes ofexemplification, show embodiments of the invention according to itsdifferent aspects.

FIG. 1 schematically shows a system in which the present invention canbe used.

FIG. 2 shows schematically a base with graphically noted commands andassociations according to a first application of an embodiment of thepresent invention.

FIG. 3 shows schematically a base with several graphically notedcommands and associations according to a second application of anembodiment of the present invention.

FIG. 4 shows schematically a base with several graphically notedcommands and associations according to a third application of anembodiment of the present invention.

FIG. 5 shows schematically a sensor device for use in connection withthe present invention.

FIGS. 6-12 are flow charts, which schematically illustrate methods forcontrolling a unit provided with a processor according to the invention.

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION

By way of introduction, the general principles of the invention will bedescribed with reference to FIG. 1. Thereafter, a number of exemplifyingapplications and alternative embodiments of the invention will bediscussed with reference to FIGS. 2-7.

The present invention is based on the general idea of controlling a unitprovided with a processor by means of commands which are written on aposition-coded base and which thereafter are sent to the unit providedwith a processor. This is illustrated schematically in FIG. 1, whichmore specifically shows a base 1 in the form of a sheet of paper, onwhich commands can be written, a sensor device 2 using which thecommands can be written on the base 1, recorded in electronic form andsent to a unit 3 provided with a processor, which in FIG. 1 isexemplified by a computer.

The sensor device 2 can communicate with the computer 3 in various ways.One alternative is for the sensor device 2 to communicate directly withthe computer 3, for example via a cable, an infrared link or ashort-range radio link, such as according to the Bluetooth standard.This is illustrated in FIG. 1 by a broken line 4. A second alternativeis for the sensor device 2 to communicate with the computer via a localor global computer network 5, such as the Internet. The sensor device 2can be connected to the computer network 5 by means of a computer 6which is permanently connected to the computer network, and with whichthe sensor device can communicate in, for example, any one of the waysmentioned above for communication with the computer 3. This is alsoshown by a broken line 4. Alternatively, the sensor device can beconnected to the computer network 5 by wireless means via a radio accesspoint 7 which is reached, for example, via a mobile telephone 8, ahand-held computer 9, such as a PDA—Personal Digital Assistant, or aportable computer 10. Optionally, these units communicate with the radioaccess point 7 via each other, for example by the portable computer 10or the hand-held computer 9 utilizing a modem in the mobile telephone 8as a link to the radio access point 7, which can be a radio access pointin some known system such as GSM, CDMA, GPRS or some other type ofmobile communication network.

As an additional alternative, the sensor device 2 can itself have meansof communication, making possible direct connection to the radio accesspoint 7.

As mentioned above, the base 1 is provided with a position-codingpattern P. The position-coding pattern P is shown only schematically inFIG. 1 as a surface provided with dots. This position-coding pattern isused to record in electronic form what is written on the base. Varioustypes of position-coding pattern which can be used for this purpose areknown. In U.S. Pat. No. 5,477,012, for example, a position-codingpattern is shown where each position is coded by a unique symbol. Theposition-coding pattern can be read off by a pen which detects theposition code optically, decodes it and generates a pair of coordinatesthat describes the movement of the pen across the surface. In WO00/73983 and WO 01/26032, which are hereby incorporated by thisreference, and both of which are assigned to the Applicant of thepresent application, another position-coding pattern is described inwhich each position is coded by means of a plurality of symbols of asimpler type and where each symbol contributes to the coding of morethan one position. In WO 00/73983 different sized dots are used to codeones and zeros in the position-coding pattern which is binary. In WO01/26032 four different displacements of a dot from a nominal positionare used to code four different pairs of bits in the position-codingpattern. A certain number of dots, for example 6*6 dots, codes a uniqueposition. The position can be calculated from the bit valuescorresponding to the dots.

The position-coding patterns in WO 00/73983 and WO 01/26032 can bedetected optically by a pen that decodes the dots and generates a pairof coordinates for each set of, for example, 6*6 dots. If theposition-coding pattern is read off while the pen is writing on theposition-coding pattern, a sequence of pairs of coordinates thatdescribes the movement of the pen across the position-coding pattern isthus obtained and thus constitutes an electronic representation of whatwas written on the sheet of paper.

In the following, it is assumed that the base 1 is provided with aposition-coding pattern of the type described in WO 01/26032. However,it should be noted that other types of position-coding patterns, for thepurposes of the invention, may have an equivalent function.

The sensor device 2 can then also be of the type described in WO01/26032. An example of the construction of such a device is describedin the following with reference to FIG. 5.

It comprises a casing 11 which has approximately the same shape as apen. At the end of the casing there is an opening 12. The end isintended to abut against or to be held a short distance from the surfaceon which the position determination is to be carried out.

The casing contains principally an optics part, an electronic circuitrypart and a power supply.

The optics part comprises at least one light-emitting diode 13 forilluminating the surface which is to be imaged and a light-sensitivearea sensor 14, for example a CCD or CMOS sensor, for recording atwo-dimensional image. Optionally, the device can also contain anoptical system, such as a mirror and/or lens system (not shown). Thelight-emitting diode can be an infrared light-emitting diode and thesensor can be sensitive to infrared light.

The power supply for the device is obtained from a battery 15, which ismounted in a separate compartment in the casing. It is also possible toobtain the power supply via a cable from an external power source (notshown).

The electronic circuitry part contains a signal-processor 16 whichcomprises a processor with primary memory and program memory. Theprocessor is programmed to read images from the sensor, to detect theposition-coding pattern in the images and to decode this into positionsin the form of pairs of coordinates, and to process the information thusrecorded in electronic form in the way described in greater detail belowfor controlling the unit 3 provided with a processor.

In this embodiment, the device also comprises a pen point 17, with theaid of which ordinary pigment-based writing can be written on thesurface on which the position determination is to be carried out. Thepen point 17 can be extendable and retractable so that the user cancontrol whether or not it is to be used. In certain applications thedevice does not need to have a pen point at all.

The pigment-based writing is suitably of a type that is transparent toinfrared light and the marks suitably absorb infrared light. By using alight-emitting diode which emits infrared light and a sensor which issensitive to infrared light, the detection of the pattern is carried outwithout the above-mentioned writing interfering with the pattern.

The device may also comprise buttons 18, by means of which the devicecan be activated and controlled. It also has a transceiver 19 forwireless transmission, for example using infrared light, radio waves orultrasound, of information to and from the device. The device can alsocomprise a display 20 for displaying positions or recorded information.

The device can be divided between different physical casings, a firstcasing containing components which are required for recording images ofthe position-coding pattern and for transmitting these to componentswhich are contained in a second casing and which carry out the positiondetermination on the basis of the recorded image or images.

According to one embodiment, the sensor device 2 communicates with otherunits 8, 9, 10 or 6, by wireless means in a way known to those skilledin the art. The communication between the units 8, 9, 10 and 6respectively, the radio access point 7, the computer network 5 and theunit 3 provided with a processor also takes place in a way known tothose skilled in the art. The function of the sensor, and theapplication of the position-coding pattern on the base are also methodsknown to those skilled in the art.

A first application of an embodiment of the present invention will nowbe described with reference to FIG. 2. On a base 1, which is providedwith the position-coding pattern, is noted a number of phrases orsymbols which constitute commands 22, 23 or addresses 21. A command canbe a word, but can also be a symbol, provided that the sensor device ispre-programmed to recognize and identify a symbol as a command. Suchpre-programming can be carried out by some form of learning, whereby acommand is associated with a symbol, as will be described in more detailbelow.

One of the noted phrases 21 constitutes an indication of a computernetwork address to which the sensor device connects, e.g. the computernetwork address where the unit 3 provided with a processor is located.The computer network address can also be some other type of address forcomputer communication, such as a Bluetooth® address.

Commands are noted on the base in a tree structure, in such a way thatan address or an indication of the address constitutes the root and insuch a way that commands constitute nodes in the tree structure. A nodeis associated with another node by a line being drawn between them.

The commands 22, 23 can thus constitute conditions, operators,parameters or subordinate commands, as shown in FIG. 2. It is recognizedthat the tree structure with commands can vary in extent, from a singlechain of commands to a large tree with many commands and subordinatecommands.

A command frame 24 may be noted around each command. The command frame24 may work in such a way that it delimits a part of the base 1 which isto be associated with the command and also in such a way that it isrecognized by the sensor device 2 and understood as an indication that acommand is being entered and thus work as a command indicator. Betweenthe commands 22, 23 or the command frames 24 lines may be drawn whichindicate connections between the commands.

The shape of the command frames in FIGS. 2-4 constitutes only oneexample of how such frames can be designed. Other shapes are possibleand different shapes can represent different types of commands oraddresses. On the base 1 there is also a “send” box 26 which indicatesto the sensor device that the commands are to be sent to the unitprovided with a processor. The “send” box 26 can be either a pre-printedbox comprising a specific, predefined part of the position-codingpattern which codes for the “send” function, or alternatively the “send”box 26 can be noted by the user on the base 1 and provided with aparticular symbol or a particular command word which represents the“send” function. As another alternative, the “send” box can be omitted,by for example the sensor device connecting directly to the computernetwork address when it identifies a command box or an indication of acomputer network address.

As shown in FIG. 2, the user has noted the address “my computer” 21 witha sensor device 2, which indicates to the sensor device 2 the computernetwork address of the user's own computer. Around the address thecommand frame 24 is noted, which defines the subarea of theposition-coding pattern which after the input will be associated withthe address by the sensor device. At the same time as the noting iscarried out, the sensor device reads off the position-coding pattern andforms an electronic representation of the graphical image that theaddress 21 and the command frame 24 constitute. The graphical image isthen interpreted using OCR or ICR, so that an electronic form of thecommand “my computer” is obtained and is stored in a memory in thesensor device. Optionally, the address is stored together with anindication that the noted “my computer” is an address and not just atext string. As an alternative, the address derived by the sensordevice, for example 197.57.3.982, can be stored. A representation of thesurface on the base 1 which is enclosed by command frame 24 is alsostored in the sensor device and associated with the address “mycomputer” as an association with an IP address, here exemplified by197.57.3.982.

When the user notes the command “format” 22 and the second command “harddisk” 23 and the command frames 24, the same procedure is repeated: thenoted commands are recorded in electronic form, interpreted and storedtogether with the indication that they constitute commands and with thesubareas of the base 1 with which they are associated. When the userthen notes associations 25 between the commands 22, 23, this is alsorecorded and stored as indications of how the commands 22, 23 arerelated to each other.

A command string is formed in the sensor device from the stored commands22, 23 and associations 25, the first component of the command stringconsisting of the address that constitutes a root in the tree-likecommand structure, that is the address “my computer”. Next associations25 are followed, until the last command “hard disk” 23 is reached,whereupon the command string is built up gradually and finally assumes,for example, the form

my computer/format/hard disk

or alternatively

197.57.3.982/format/hard disk

Command strings may be formed and stored in different ways. According toone alternative, each command string that may be formed from a givencommand structure or hierarchy may be stored. According to thisalternative, new command strings are added as new commands or parametersare added to the command structure.

According to another alternative, a tree structure that has been notedgraphically and registered may be represented in any appropriate way inthe sensor device, such as by means of any data structure forrepresenting tree structures. When a certain command is selected, bye.g. registering a pair of coordinates within the command frame on thebase, the corresponding command string is formed from the marked commandor parameter and all other commands or parameters through the root.

When the user marks the “send” box 26, the command string is sent to thecomputer network address that is indicated by the address “my computer”,whereupon the unit 3 provided with a processor and connected to thecomputer network address executes the command and its hard disk isformatted.

In one embodiment of the present invention, the unit 3 provided with aprocessor then sends an acknowledgement to the sensor device 2 that thecommand has been carried out. The acknowledgement can be presented tothe user in the form of a sound, light or vibration signal in the sensordevice or by being displayed on some other unit in the vicinity of theuser, such as a mobile telephone or hand-held computer.

A second application of an embodiment of the present invention will nowbe described with reference to FIG. 3. On a base 1 provided with aposition-coding pattern, an address 30 and a number of commands 31, 32,33, 34, 35, 36, surrounded by command frames 24 and connected byassociations 25, have been noted in a similar way to that describedabove with reference to FIG. 2. The commands have been noted and linkedtogether into a tree-like structure, where an address “home” 30indicates a computer network address, e.g. of a unit which is situatedin the user's intelligent home, which unit is arranged to control one ormore units provided with a processor in the user's home. Alternatively,each unit provided with a processor in the user's home can have anetwork address and can be connected directly to the computer network,without any master unit as in the example.

A number of units provided with a processor are connected to thecomputer network address, which units are addressed by the commands 31,32, 33, 34. Other types of units are possible and are to be regarded ascovered by the invention. Each unit can then be provided with a numberof commands or parameters for its control. In FIG. 3, the tree structureis drawn out in full only for the temperature control of a heatingsystem, but it is recognized that the structure can be extended as newunits are added, or as existing units are provided with new functions.It is also recognized that the tree structure shown in FIG. 3 is drawnmechanically and that for the purpose of illustration it shows a largerpart of the tree structure than what is necessary for the example. In anactual application the tree structure or command hierarchy may behand-drawn, and only that part of the structure, that is those commands,which are to be used are drawn, as in FIG. 2. Alternatively, predefinedand printed or machine drawn command structures, or parts thereof, areconceivable. These may e.g. be provided by the manufacturer of a certainhousehold appliance. Such predefined command hierarchies may beexpandable by e.g. allowing the user to add further parameters orcommands. It is also possible that the manufacturer of a householdappliance provides a complete, predefined command hierarchy containingall the necessary commands or parameters for an appliance or a group ofappliances. The command hierarchy may be provided in the form of acontrol base, e.g. a sheet or a paper, provided with the position-codingpattern, on which the command hierarchy is preprinted, but where theaddress has not been filled out. A corresponding electronic version ofthe command hierarchy may be provided in the form of software forinstallation in the sensor device. The software provides the necessaryinstructions for the sensor device to associate the subareas of theposition-coding pattern with the proper command.

The user may install the appliance by filling in his address with thesensor device, at the proper position in the command hierarchy, thusmaking the appliance controllable by means of the sensor device. Oncethe command hierarchy has been installed in the sensor device, the usermay use it and add commands according to what has been described above.

Thus, the appliance manufacturer may provide its customer with a productkit comprising the control base and software for installation of thecommand hierarchy in the sensor device.

A command 35 which indicates the temperature parameter and a valuecommand 36 for this are marked and connected by means of an associationline 25. The command string which is generated for setting thetemperature of the heating installation is home/heating/temperature/20,

whereupon the heating installation in the user's home sets thetemperature to 20 degrees, in response to the user sending the commandstring to the computer network address, which is indicated by theaddress “home” by marking the node containing the temperature parameterand then the “send” box 26. It is recognized that similar commandstructures can be constructed for all the other units 31, 32, 33, 34,whereby setting parameters and values can be entered and sent to theunits 31, 32, 33, 34.

FIG. 4 shows a third application of an embodiment of the presentinvention. According to this application, the input is carried out toprograms in the user's computer 3 (FIG. 1). Also in this case, anaddress 40 and a number of commands 41, 42, 43, 44 have been drawn on abase 1 and connected by associations 25 and marked with command frames24. A “send” box 26 is also arranged on the base 1. A sketch 45 has alsobeen noted on the base and recorded in electronic form. The commandstructure in FIG. 4 comprises an address “my computer” 40 whichindicates the address of the user's computer, and a graphics program 41,a word processing program 42 and a spreadsheet program 43 which areinstalled in the user's computer 3.

In this example, the input of data in the form of a sketch to thegraphics program 41 is described, but it is recognized that input to theword processing program 42 and the spreadsheet program 43 can be carriedout analogously. Practically all information that can be noted on a basecan also be transferred in this way. Other examples are calendarinformation, memos, database entries, etc.

As a subordinate command to the graphics program, the command “import”44 has been noted and recorded in electronic form using the sensordevice. The sketch 45 is connected by a line 47 to the command “import”44. The sketch 45 is stored in the sensor device in the form ofgraphical data, below called “image data”. More specifically, the sketchis stored as a graphics file, e.g. a vector graphics file. This can bein a standard storage format such as .wmf (Windows® Meta File) or in astorage format specific to the sensor device. The file is transferred tothe unit provided with a processor before the command is executed or inassociation with the command being executed.

In response to the “send” box being marked, the following command stringis sent to the unit 3 provided with a processor:

my computer/graphics program/import/image data.

The unit 3 provided with a processor receives the command string and theimage and causes the graphics program to import the image. When astandard storage format is used, the importing to the graphics programcan be carried out by the program's existing capability of beingexecuted by indicating a command string comprising a file name. However,if a storage format specific to the sensor device is used, it isnecessary for the program that receives the image data to have beenprovided with functionality for handling the storage format of thesensor device.

It is recognized that additional commands to the graphics program can benoted on the base 1 and used for more precise control of the input ofthe sketch 45. As an alternative, the area that is to constitute imagedata can be marked in order to delimit it from other areas that are notrequired to be transmitted to the unit 3 provided with a processor. Sucha mark 46 is shown in FIG. 4.

A base 1 on which commands are noted can be used repeatedly by thecommands being stored in a memory in the sensor device when the notingis carried out. This may mean that the user can indicate to the sensordevice which command is to be carried out by pointing at a written-downcommand with the sensor device so that the sensor device can read offthe position-coding pattern corresponding to the command. The sensordevice may identify this command in its memory and send it to theaddress with which the command was associated when the noting on thebase was carried out. This may be done by simply recording a pair ofcoordinates within the proper subarea of the position-coding pattern,without marking any “send” box.

According to one embodiment of the invention, further commands orparameters defining a previously noted and recorded command may be addedby being noted graphically on the base and associated with the commandin question. In this way, a dynamic command structure is obtained, whichcan be enlarged as new units are added or enlarged with new commands.Commands that has been added to the command hierarchy in this mannerwill thus constitute parts of the command hierarchy as described above.

The methods described here may be implemented as a computer programproduct, as shown in FIGS. 6 and 7. The computer program productcomprises a computer program which is stored in the program memory ofthe sensor device and is executed in its processor. As an alternative,the method can be implemented completely or partially in the form of aproduct-specific circuit, such as for example an ASIC, an FPGA or in theform of digital or analogue circuits or in any suitable combination ofthese.

The following description is directed to the inventive method based onFIGS. 6-12, which show different embodiments of the invention, which maybe used separately or in combination.

FIG. 6 is a schematic flow chart for a method according to of theinvention. The method may be implemented in e.g. a computer programproduct. A graphical notation is received in step 50 in the sensordevice 2. A command for the unit 3 provided with a processor isidentified in step 51 based on the graphical notation. The sensor devicealso receives in step 52 an address to the unit 3 provided with aprocessor. In step 53, the command is sent to the address. Step 53 maybe initiated in different manners, by e.g. the sensor device 2 detectinga “send”-box on the base, by receiving a send command or any otherindication such as a button being depressed etc.

FIG. 7 is a schematic flow chart for a method according to an embodimentof the invention. In FIG. 7, the step 51 for identifying the command forthe unit 3 provided with a processor, comprises a substep 51 a of atleast partly converting the graphical notation into a machine readablecharacter format by e.g. ICR (intelligent character recognition) or HWR(handwriting recognition), based on the output from step 51 a, whereuponthe command may be identified in step 51 b. Thus, a command that iswritten in plain text may be identified, either directly from the actualcombination of machine readable characters, or by retrieving a relatedcommand from a database on the basis of the character combination.Likewise, the sensor device may be arranged to identify an address fromsuch a character combination (step 52).

FIG. 8 is a schematic flow chart for a method according to anotherembodiment of the invention. In FIG. 8 the step 51 of identifying thecommand comprises a first substep 51 c of identifying a graphicalsymbol, which may be predefined, and thus recognized, as beingequivalent to a certain command. In a second substep 51 b, thecorresponding command is identified. For example, a cross mark (“X”)could be interpreted as a “delete” command. Likewise, the sensor devicemay be arranged to identify an address from such a graphical symbol(step 52).

Evidently, the embodiments of FIGS. 7 and 8 may be combined, by thesensor device being capable of first determining whether a command oraddress is recorded in plain text or not, and then identifying thecommand or address based on either a character combination or agraphical symbol.

Further commands may be added in additional steps (not shown)corresponding to those described in FIGS. 6-8.

FIG. 9 is a schematic flow chart for a method according to yet anotherembodiment of the invention. In FIG. 9, the step 51 of identifying thecommand further comprises the substep 51 d of detecting a commandindicator, such as the command indicators 24 of FIGS. 2-4. In connectionwith the detection of the command indicator, a subarea of theposition-coding pattern may be identified in step 51 e. The subarea maybe associated with the command in step 51 f, such that a recording of apair of coordinates within the subarea will be interpreted by the sensordevice as being equal to a recording of the associated command.

FIG. 10 is a schematic flow chart for a method according to a furtherembodiment of the invention. In FIG. 10 the step 52 a of receiving theaddress comprises receiving the address from a memory 55, which may beincorporated in the sensor device 2.

FIG. 11 is a schematic flow chart for a method according to anotherembodiment of the invention. In FIG. 11, the command (in step 51), theaddress (in step 52 b) and the association (in step 54) are allidentified based on the graphical notation 50.

FIG. 12 illustrates an embodiment of a method according to theinvention, in which a command hierarchy in step 57 has been provided ina memory 56 of the sensor device 2. The command hierarchy may beprovided by storing graphical notations that has been made at earlieroccasions. It may also be provided by downloading into the memory 56from e.g. an external memory medium, e.g. via any one of thecommunication paths discussed above in relation to FIG. 1. In the memory56, commands may be e.g. associated with subareas of the position-codingpattern.

In FIG. 12, the step 51 of identifying the command may comprise asubstep 51 g of receiving a pair of coordinates from a subarea that isassociated with the command. Thus, in substep 51 h, the commandassociated with the subarea is identified.

FIG. 12 also illustrates that based on the command hierarchy, commandstrings may be formed. The command strings may be formed while commandsare being entered through graphical notations or in response to acommand being identified through e.g. a graphical symbol or through apair of coordinates within a predefined subarea.

Different procedures for preprogramming or predefining addresses and/orcommands are conceivable.

One alternative is to use a special purpose form provided with aposition-coding pattern, where the user, in e.g. dedicated boxes, writesa short name, an address or a command that is to be made identifiablefor the sensor device.

Another option is to perform the preprogramming by means of a unit thatcommunicates with the sensor device, such as a computer, a PDA etc. Sucha unit may communicate via e.g. a computer network or via short rangecommunication such as Bluetooth® or IrDA.

From the above description, it should be apparent to the person skilledin the art that different embodiments of the method according to theinvention may be combined and that the steps may be performed indifferent order.

The graphical notation may also comprise e.g. data in the form of e.g.figures, text, sketches or graphics, which is recorded by the sensordevice 2 and associated with commands or addresses as described abovewith reference to FIG. 4.

The sensor device may be programmed to evaluate the graphical notationsas they are received, based on their contents. For example, a plain textcommand, a graphical symbol or a command indicator may trigger the abovedescribed method of identifying commands. All other, non-recognized datamay be treated in any standard fashion, such as may be stored asstrokes, i.e. sequences of coordinate pairs, in the memory of the sensordevice 2.

The invention can also be varied in other ways within the scope of theappended claims.

For example, many different types of position-coding pattern areconceivable, in addition to those shown herein. The position-codingpattern does not necessarily need to be optically detectable. It could,for example, be readable by magnetic, capacitive, inductive, chemical oracoustic means. However, this would require a different type of sensorto be used.

The position-coding pattern in WO 01/26032 can code coordinates of avery large number of unique positions or points. It can be considered asthough all these points together make up an imaginary surface which isconsiderably larger than any single base. This imaginary surface can bedivided into different areas which are reserved for differentapplications. An area can, for example, be reserved for controllingunits provided with a processor. Information defining such areas andfunctions connected thereto can be stored in the pen and utilized forcontrolling the function of the pen.

Another alternative is to allow commands to be noted within practicallyany part of the imaginary surface. In this way almost any surface whichis provided with a position-coding pattern can be used for enteringcommands and for the associated control of a unit provided with aprocessor. In this embodiment measures may possibly need to be taken toprevent interference, as certain areas can previously have been reservedfor certain functions.

The address to which commands and data are sent can be identified by,for example, looking up addresses in a database external to the sensordevice. Both commands and address could be sent to an external unit forinterpretation and further processing. This interpretation can becarried out in the sensor device, in the unit provided with a processor,or in some other external unit, possibly dedicated to the purpose.

It is further possible to determine in advance the address to which thecommand string is to be sent, for example, by the sensor device beingpre-programmed with such information. According to this embodiment, itis not necessary to note an address graphically. At least one commandcan be noted, but command structures as described above can also benoted and stored according to this embodiment. Alternatively, theaddress to which the command string is to be sent may be associated witha specific area of the position-coding pattern, and thus with a specificbase on which that area of the position-coding pattern is arranged.

As an alternative to storing the commands as a dynamic structure, it ispossible to store a plurality of command strings where each commandstring represents a conceivable combination of commands.

The form in which the commands and the address are sent can also vary:it is, for example, possible to send raw data in the form of the imageswhich the sensor device takes of the base. It is also possible to sendsome form of processed, for example compressed, image data, a series ofcoordinates which has been derived from the images and which representsthe movement of the sensor device across the base, or commands oraddress in character-coded format. Other ways of sending commands,addresses or data are not excluded.

It is also possible to permit association of commands and/or data thatare on different bases. This could, for example, be carried out by anaddress being noted on a first base and a command being noted on asecond base, then the address and the command are linked by the basesbeing arranged next to each other, after which a line is drawn betweenthe address and the command. The discontinuity in the position-codingpattern which then probably arises, can be handled by the sensor device,e.g. by creating an association between the two areas of theposition-coding pattern on the respective side of the discontinuity,such as is described in WO 01/75781, which is hereby incorporatedthrough this reference.

The sending of address, commands and data to the unit provided with aprocessor can be initiated by the use of the “send” button, but can alsobe initiated in response to an indication being written down on thebase. For example, the drawing of a frame around a command can initiatetransmission. It is also possible to initiate transmission as soon asthe sensor device has recorded a complete command string, or when apartial area within a frame is marked which surrounds an alreadywritten-down command. Further alternatives for initiating sendingcomprises, but is not limited to voice control, depressing a button onthe pen etc.

In addition, entry of addresses and commands can be marked by, forexample, the user pressing a button on the sensor device. This canprecede the entry of a command, but the button can also be helddepressed during the whole or part of the entry procedure.

Other variations and combinations are also possible within the scope ofthe appended claims.

1. A method for controlling a unit provided with a processor,characterized by: receiving at least one graphical notation in the formof positions representing a sensor device's movement across a base thatis provided with a position-coding pattern, while the graphical notationwas made, identifying, based on said at least one graphical notation, atleast one command for said unit provided with a processor, receiving anaddress to the unit provided with a processor, and controlling the unitprovided with a processor by sending said at least one command to theaddress.